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Abstract 

We introduce the entropy rate of multidimensional cellular automata. 
This number is invariant under shift-commuting isomorphisms; as op- 
posed to the entropy of such CA, it is always finite. The invariance 
property and the finiteness of the entropy rate result from basic results 
about the entropy of partitions of multidimensional cellular automata. 
We prove several results that show that entropy rate of 2-dimensional 
automata preserve similar properties of the entropy of one dimensional 
cellular automata. In particular we establish an inequality which involves 
the entropy rate, the radius of the cellular automaton and the entropy of 
the d-dimensional shift. We also compute the entropy rate of permutative 
bi-dimensional cellular automata and show that the finite value of the en- 
tropy rate (like the standard entropy of for one-dimensional CA) depends 
on the number of permutative sites. Finally we define the topological en- 
tropy rate and prove that it is an invariant for topological shift-commuting 
conjugacy and establish some relations between topological and measure- 
theoretic entropy rates. 



1 Introduction 

A cellular automaton (CA) is a continuous self-map F on the configuration 
space A 1, , commuting with the group of shifts on this space. CA are sim- 
ple computational devices for computer scientists and they are nice models for 
physicists. Mathematicians view them as an interesting family of topological 
and measurable dynamical systems. 

The entropy of a CA map F acting on some full shift A z , in its measure- 
theoretic as well as its topological versions {h^(A z ,F) and h(A z ,F) respec- 
tively) is an important measure of the local unpredictability of the map. Each 
of the two entropies is an invariant under the suitable kind of conjugacy. 
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The entropy of l-dimensional CA is always finite. But when d > 1 this 
measure is a crude one. Already in the two-dimensional case the entropy of 
a cellular automaton is often infinite. This is true for whole families of CA, 
the dynamics of which is especially tractable. For instance it is shown in [3 
that for the class of additive two-dimensional CA on {0, 1} Z , which may be 
seen as a subclass of two-dimensional permutative CA, defined in Section [51 
the entropy is alway infinite. It was conjectured by Shereshevsky that for a 
two-dimensional CA the entropy could be or infinite. In jd] Meyerovitch has 
shown that there exist non-trivial examples of two-dimensional CA with finite 
positive entropy. To finish with the entropy of two-dimensional CA, we can say 
that it look impossible to establish some inequalities between the entropy of the 
automaton and the entropy of the group of shifts since for this last value we 
need to divide by some square of the number of iterations (see definitions done 
by equality [5] ) . 

Here we introduce entropy rate for CA acting on A z . It is not hard to obtain 
similar results for CA on A z , d > 2, with proper changes in the definition of 
entropy rate. It is derived from partial values of the entropy of the CA and can 
be expressed as follows for an F-invariant measure [i which is also invariant for 
the group of shifts: 

ER M ( A 2 ' , F) — lim sup —h^(S n , F), 

where S n is the clopen partition of A 1,2 according to the values of the coordinates 
in the square of side 2n + 1 centred at the origin. It is finite for any CA. It 
is very deeply grounded in the shift structure of the configuration space; as a 
consequence it is mostly significant when fi is also invariant under the group 
of shifts, and in this case it is an invariant for shift-commuting isomorphisms. 
Note that limsupyj^^ i/i At (5 ra , F) defined for all F-invariant measure /j, is an 
invariant for continuous and shift-invariant isomorphisms only (see subsection 
I3.1[) . The topological entropy rate 

ER(A z2 ,F) = lim sup -h(S n ,F) 

n— >-oo 

has similar properties and similar limitations. 

One could define the entropy rate of one-dimensional cellular automata: it 
is equal to their usual entropy, up to some multiplicative constant, and does 
not bring any further information about the dynamics. On the other hand, the 
entropy of a CA in higher dimensions is often infinite, whereas its entropy rate 
is always finite, like the entropy in one dimension, so entropy rate turns out to 
be more sensitive than entropy when d > 2. In particular, it makes it possible 
to obtain inequalities, as shown in Section U and [3] 

Let A be a finite set of cardinality #A We denote by A 1 , the set of 
configurations or maps from 1 d to A. In this paper we mainly restrict our 
study to the case d = 2. We note that A z endowed with the product topology 
of the discrete topologies on the sets A is a compact space. Let £ be the group 
generated by the the d shifts <Xj (1 < j < d). 

Note that it is possible to generalize the Curtis-Hedlund-Lyndon theorem 
(see [4]) and state that for every cellular automaton F there exists an integer 
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r called the radius of the CA and a block map / from A^ 2r+1 ^ to A such that 
F(x(ii,. . .,i d )) = f(x([ii -r,ii + r],..., [id -r,i d + r]). 

Entropy The entropy (metrical (h^T)) or topological h(T)) is an isomor- 
phism invariant that measures the complexity of the dynamical system (X, jj,, T) 
or (X, T) . For each one-dimensional cellular automaton F of radius r it is well 
known that h^F) < h(F) < 2rln(#A). In the ergodic setting (for the shift or 
the CA F) it was shown (see [5]) that h^F) < (A+ + A~) • h^a) < 2r ■ h^(a) 
where a is the shift on A z and A are discrete Lyapunov exponents. In Propo- 
sition [5] we show that the last inequality (F) < 2r ■ (a) remains true for 
shift and F-invariant measure y, for the one-dimensional case. There exist some 
strong relations between dynamical properties of the CA like equicontinuity and 
the fact that the entropy is equal to zero (see pQ and [TO])- Is there exists similar 
results for the entropy rate of two dimensional CA? In the class of permutative 
one-dimensional CA the entropy rate is easy to compute. For instance when F 
is a CA of radius r permutative in coordinates —r and r the value of the en- 
tropy is h(F) = 2r x ln(#^4). For two dimensional permutative CA, the entropy 
hn(F) = +00. 

The Variational Principle (see for instance [12]) which states that h(F) = 
sup^ hf_t (F) implicitly introduces the question of the existence of a set of mea- 
sures of maximum entropy: may it be empty? May it contain more than one 
measure? As far as we know those questions are open even when d = 1. Note 
that for the permutative class this set is not empty and contains the uniform 
measure. 

In this paper we introduce a formal definition of the entropy rate that is 
derived directly from the definition of the entropy. A first tentative and incom- 
plete definition of measurable entropy rate was given by the second author in 
[9] as a draft; a little later in [5] Lakshtanov and Langvagen introduced some 
similar notions for the topological case. None of those two definitions allow to 
prove invariance under some class of isomorphisms. 

New definition and results 

In this paper we introduce the notion of entropy rate of partition V de- 
noted by ER IJi (V,F) and define the measurable entropy rate ER^(A Z ,F) as 
the supremum over all the finite partitions of the entropy rate of a partition 
(see Definition [TJ [2] and [3]) . Using some particular properties of the entropy of 
bi-dimensional cellular automata (see Lemma [1} we show in Proposition [5] that 
there exists a partition So such that ER(A ,F) = ER^Sq, F) when /i is an 
^-invariant and shift commuting measure and establish in Proposition [T] that 
the the entropy rate is finite (ER t _ l (A I? ,F) = ER^(Sq,F) < 8rln(#A)). 

Next we show that for an F and shift-invariant measure the entropy rate 
denoted by ER^ (A z , F) is an invariant for the class of shift commuting iso- 
morphism (see Proposition . In Subsection 13.11 we prove that entropy rate of 
the partition Sq: ER^(Sq, F) is an invariant for continuous and shift-invariant 
isomorphism for all _F-invariant measure u. 

H7 2 nil 

We also prove that for any CA F : A — > A of radius r permutative at the 
four sides of the square E r used to define the local rule / (see Definition [SJ we 
can compute explicitly the entropy rate and obtain ER^ X (A z ,F) — 8r ln(#^4) 
where /i\ is the uniform measure on A z . When there is less than 4 sides 
of the square E r with permutatives points we compute the entropy rate for 
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the subclass of additive cellular automata and show that the entropy rate is 
proportional with the number of permutative points (see Proposition [11]). This 
result could be compared with the entropy of additive one dimensional C A where 
there is also a proportion between the entropy and the number of permutative 
points (see [3]). 

Moreover we also note that the uniform measure on A z is a measure of max- 
imum entropy rate for the classe of permutative CA whereas the uniform mea- 
sure on A z is a measure of maximum entropy for permutative one-dimensional 
CA. More generaly we show in Theorem [1] that for any bi-dimensional cellular 
automaton F and measure \i invariant by F and by the group of shift E on 
A 1,2 we have ER^(A z2 ,F) < 8r ■ h^(A z2 ,a) where h^(A z2 ,a) is the entropy 
of the two-dimensional shift. This result could be compared with the fact that 
h l j,(A I ',F) < 2r ■ h/j,(A ,&) proved in Proposition [5] with the same setting for 
the measure. We note that the last inequality is optimal in a sense that it is 
an equality in the permutative case and that it is not possible to establish an 
analog one linking the entropy of the two dimensional shift and the entropy of 
the CA. Moreover the proof requires the use of many properties of the entropy 
and conditional entropy. 

In Section [6] we introduce the topological entropy rate and show that like the 
measurable entropy rate, it is finite (Proposition I12[) and that it is an invariant 
for shift commuting homeomorphisms of A z (Proposition [15]). Next we show 
that for all positive integer k > 1 one has ER(A Z ' 2 ,F) = k ■ ER(A z2 ,F). This 
property is also shared by the entropy and the measurable entropy rate. Then 
we give a relation between the two entropy rate showing (see Proposition [T7] ) 
that 

ER(A Z ',F)> sup {ER fi (A z ' 2 , F)} 

HeM(F,a) 

and 

ER(S ,F)> sup {ERn(So,F) 

where M(F) is the set of F-invariant measures and M(F, a) the subset of M(F) 
of measures invariant for the group of shift £ on A z . 

Another result shows (see Proposition [T^| that topological entropy rate de- 
pends mainly on the local rule of the CA and not on the dimension of the CA 
space. More precisely when a CA acts on a two-dimensional space but its block 
map can be reduced to a one-dimensional one, its topological entropy rate is 
equal (up to some multiplicative constant) to the entropy of the corresponding 
one-dimensional CA. 

All the presents results seem to show that entropy rate is a rather well 
extended notion of entropy for multi-dimensional cellular automata and could 
be used to make progress in the understanding of these particular dynami- 
cal systems. Some drawback could appear, for example the definition use a 
limit superior (ER^(Sa, F) = limsup,^^ —h^Sn, F)) instead of the entropy 
that appears like a simple limit. Nevertheless the entropy rate of permuta- 
tive CA came from a limit (see Remark [5] and Proposition [TU| and the values 
limsupn^^ —h^(S n ,F) and liminfn^oo —h^SnjF) differ only no maximum of 
a factor 8 (see Proposition [4] and Proposition [16] (ii) for the topological case). 
Moreover this last property gives more meaning to the properties ER^ (F) = 
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and ER(F) = that could be linked with some dynamical properties of the two 
dimensional CA as it occurs for the properties h ll (F) — and h(F) = (see for 
instance [J, and [TU]). 

Note that those results (for the topological and measurable case) can easily 
be extended to dimensions higher than two using more complex notations. 

2 Definitions and background 

2.1 Symbolic spaces and cellular automata 

Let A be a finite set or alphabet) its cardinality is denoted by #A. For an integer 
d > 1 let A z be the set of all maps x: 1 d — > A; any such map x £ A z is called 
a configuration. Given a finite subset C of Z d , one defines a pattern on C as 
a map P:C —> A, in other words, an element of A c . When d > 1 the usual 
concatenation of words can be extended to some patterns in the following way: 
given C, C C Z rf such that C n C = and two patterns, P on C and P' on C, 
the pattern ?»P'onCUC"is the one such that (P • P'){z) = P(z) for z £ C 
and (P • P'){z) = P'{z) for z £ C . Again for C C Z d , the pattern x c is just 
the restriction of the map x to the set of coordinates C. 

The configuration space A 7 " is endowed with the product of the discrete 
topologies on the various coordinates. For this topology A z is a compact 
metric space. For z — (i,j) £ Z 2 put \z\ = \/i 2 + j 2 . a metric compati- 
ble with this topology is defined by the distance d(x, y) = 2~ h where h = 
min{|z| such that x z ^ y z }. The shift maps a % ^:A ljd ~ > A z , i,j € Z are 
defined by a h: >(x)k,i — (xk+i,i+j), k,l e Z. For ! e Z and v = (i,j) G Z 2 
put t.v — (ti,tj). The shift maps form a group. It is worth while to consider 
this group of shifts S = {cr t '-'\i,j € Z} as acting on A z ; the dynamical system 
(A z , E) is often called the full shift of dimension d. 

All probability measures \i on A z that we consider are defined on the Borel 
sigma-algebra B generated by the topology of A z . 

The Curtis-Hedlund-Lyndon theorem states that for every cellular automa- 
ton F there is a finite set C C Z d and a map / from the set of patterns on C 
to A such that for z € Z d one has F(x) z = f(xc+z)', f is called the Zoca/ map 
of the CA F. One easily sees that equivalently there exist r £ N, E r being the 
square centered at the origin of size 2r + 1 and a map / from the set of patterns 
on E r to A with the same property. This is the form we are going to use. In 
this case the integer r is called the radius of F. Recall that the uniform measure 
on A z is invariant under a cellular automaton F, i.e., fio F = fi, if and only if 
F is onto [4]. 

2.2 Entropy 

Given some probability space (X, A, /x) let F(X) be the set of all finite A- 
measurable partitions of X. If V = {Pi, . . . , P n } and Q = {Qi, ■ ■ ■ , Q m } are 
two measurable partitions of X, denote by V V Q the partition {Pi n Qj ; I < 
i < n; 1 < j < to}. If for all I < i < n there exists a subset J C [1, . . . , to] such 
that Pi — Uj e jQj we write that V A Q. 
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Put H^V) = Ep e p A*( p ) log//(P). H„ is sub-additive, that is, H^WQ) < 
B^ + B^Q). Whenever V, Q G F(X) and V < Q one has < H^Q). 

By ([fl Theorem 4.3]) 

(?) H M (P V Q/TZ) = H^V/K) + E^Q/V V TZ < H„(V /K) + B„(Q/ V 1Z) 
(it) B^(V V Q) = + H^Q/V) < H^V) + B^(Q). 

(1) 

Let T be a measurable transformation of X leaving invariant: /ioT = /j. 
The entropy of the partition V with respect to T is defined as h^i^P, T) — 
lining ^H^V^T-'-p). Remark that h^(V,T) is well-defined because by 
sub-additivity of B^ the sequence ^H^V^T-^V)) is non-increasing with 
n; in particular this implies that h fi ('P,T) < B^V). Finally the entropy of 
(A,2» is h M (T) = sup PeF(x) h^(V,T). Recall that H^T^V) = B^V) and 
by QU Theorem 4.12] 

/i jU (Q ) T)</i jU (P,T) + ^(fi|P). (2) 

An isomorphism between two measure-theoretic dynamical systems (A, A, n, T) 
and (A', A', fjf, T') is a 1-to-l, bi-measurable map ip between two sets E € A 
and E' G A' such that fi(E) = Li'(E') = 1 and that p o T = T o ip on the set 
E. When such a map exists h^iT) = /i^/(T"), in other words the entropy is 
invariant under isomorphisms. 

Now for the topological setting. If U, V are open covers of a compact space 
X their join U V V is the open cover consisting of all sets of the form An B 
where A G U and B G V. An open cover U is coarser than an open cover V, 
or W =3 V, if every element of V is a subset of an element of U. If U -< V and 
W' r< V thenW VW' =S VV V. 

When U is an open cover of X, put B(U) = hx(N(U)), where N(U) denotes 
the smallest cardinality of a finite subcover of U. Like B^ the function B is 
sub-additive, in this case, B(U V V) < B(U) + H(V). Whenever V A U one has 
F(V) < ff(W). 

Let T be a surjective continuous map of X. By sub-additivity of B the 
sequence -^B (V^qT -1 (Li)) is non-increasing with u; the topological entropy of 
the cover U with respect to T is defined as ft(W, T) = lim„^ 00 ^(V^T-^W)) 
and the entrop?/ of (A,T) is h(X,T) = sup u h(U,T) on the set R(A z2 ) of all 
finite open covers of A. When U is an open cover h(U, T) < B(U); when V <U 
are two open covers one has h(V,T) < h(U,T). Another important inequality 
is 

h(UVV,T) <h(U,T) + h(V,T). (3) 

Of course topological entropy is invariant under (topological) conjugacy, that is, 
if if: (A, T) — > (A', T') is a one-to-one continuous map such that ip o T = T' o ip } 
then h(X,T) = h(X',T'). 

3 Entropy rate for a measure 

Here we define the entropy rate of a cellular automaton F for an P-invariant 
measure fi. Then some of its basic properties are explored. 

We first introduce two families of finite subsets of 1? (E n was less formally 
introduced in the first Section): 
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Definition 1. E n C Z 2 is defined to be the square of size 2n+ 1 centred at the 
origin: E n = {v = G Z 2 | — n < i, j < n} . 

For n > r, where r is the radius of the CA, E' n is the outer band of width r of 
E n : E' n = E n \ E n - r . 

To a finite measurable partition V G F(A Z ) one associates two other finite 
partitions with the help of E n and E' n ,: 

Definition 2. ForV G ¥{A I? ) one defines 

V n = V ffV ^ ifomeH) 
veE n 

and 

K = \/ a v (V) (forn>r). 

When setting V = So, where So is the clopen partition according to the 
value of the Oth coordinate, one has a particular expression for (So) n , which we 
denote by S n : 

S n = \/ a v (S Q ) = ({x e A z2 | x\ En = c} | c G A E "). 

veE n 

Likewise put 

S' n = \/ o»(So) = ({x G A^ | x\ K = c} | c G A E ' n ). 

v£E' n 

The partitions V n and V n have been introduced here in their general form 
for proving Propositions [5] and [3J Apart from this technical use we do not un- 
derstand their meaning well. In the sequel we use them mostly in one particular 
case, when V = Sk or S' k for some k; in this case they are clopen partitions 
according to local patterns, a classical tool in symbolic dynamics. 

Two properties of the partitions S n , n G N do not hold for the partitions S' n : 
by the definitions (Si)j — Si+j; and the partitions S n , n G N generate increasing 
algebras that converge to the Borel cr-algebra on A 1,2 . The last propertv implies 
in particular that if F is a CA and /z is an F-invariant measure on A z one has 
h^A 2, ,F) = lim n _j.oo h ll (S n ,F) [12 . As S n is also an open cover of A z , and 
since for any finite open cover U there is N such that U =3 Sn, one also has 
h(A z2 ,F) = limn-voo h(S n ,F) |H]. 

Definition 3. Let F be a cellular automaton on A z with radius r, and let \x 
be a probability measure on A z , invariant under F. If V is a finite measurable 
partition of A 1, , its entropy rate is 

ER^V, F) = limsup -h^V^F); 

n— >-oo ^ 

the entropy rate of the dynamical system (A z , F) endowed with the measure fi 
is the non-negative real number 

ER^.F) = mp{ERn(V,F) | V G F(A z2 )}. 
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The first step for investigating entropy rate consists in remarking that en- 
tropy rate is the same for partitions S n and S' n , and also the same for S n and 

S m , TO ^ 71. 

Lemma 1. Let F be a cellular automaton with radius r acting on A , and /i 
be an F -invariant measure. 

(i) Whenever n > r one has 

h ll {S' n ,F) = h ll {S n ,F), 

(ii) for n > r and to £ N one has 

ER^S, U F) = ER^S'^F) and ER„(S m ) = ER^(S ,F) 
Proof, (i) By the definition of entropy and since S„ — S' n V <S„_ r , 

II.. [ 

jv->oo TV 

\ l=U 4=U / 

(4) 

Because F is a cellular automaton with radius r, the vtli coordinate of F(x), 
v E Z 2 , is determined by all coordinates of x that are within the scjuare E r -\- v. 
In particular all coordinates of F(x) in £„_ r are completely determined by the 
coordinates of x in E n — E' n U E n - r , that is to say, S n - r ^ F^ 1 (S' n V S n _ r ) 
and more ge nerally F~*(«S n _ r ) A ~ 1 (S' n V <S n _ r ) . Applying F^ 1 inductively 
and using this remark each time one gets 

N-l N-l N-l 

V F-'(S' n ) \/ F- i (S n - r )= V F-\S' n )VF- N +\S n - r ). 

i=0 i=0 i=0 

Inject this simpler form into (jU) and then apply (jTJii) ) . This yields: 



/N-l N-l > 

h„(S n ,F) = h»(S' n \/S n - r ,F) = lim -H, \/ F-\S' n ) \J F-*(5 n _ r ) 



hence 

M-5n,-F) < M^) + lim ^-H^{F~ N+1 (S n - r )). 

iV— >oo iV 

Now since /j, is F- invariant the real number H^(F^ N+1 (S n - r )) = ff M («S n _ r ) 
if does not depend on N, so that in the end 



hfj,(S n ,F) < h^S' n , F) + lim = h^(S^,F). 

N— ^oo iV 



The reverse inequality is obvious since <S^ =3 iS„ . This establishes the first claim, 
(ii) Fix i > 0: because of the obvious identity (So), = Si one has 

ER^SuF) = limsup -h,j,(S n+i ,F) — lim sup — ^—h fl (S n+i , F) = ER^So, F). 

n—>oo n n— >oo n. + z 

Using (i) the equality ER^Sm, F) — ER^(S' m , F) immediately follows. □ 

With the help of this Lemma one shows that the entropy rate of the 'square' 
partitions S n is finite and does not depend on n. 
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Proposition 1. For any cellular automaton F acting on A z , any F -invariant 
measure \x, any i > 

ER„(Si,F) = ERn(So,F) < 8rlog(#A) < oo. 

Proof. By LemmaQJi), since S' n — \/ veE , a v (So), and by Jljii)) 

ER ll (S ,F) = Kmsa-p -hp(S' n ,F) = limsupi/i M ( \f (a v (S ,F)) 
<limsnpi ]T h^a v (S ),F). 

n— >oo ri 

Now as a v (So) is the partition according to the coordinate v, by elementary 
upper bounds one gets 

h^(a v (S ),F) < if>»(5„)) < log(#A). 

Combined with the above upper bound for ER^Sq, F) and since the cardinality 
of E' n is less than or equal to 8rn this yields 

ERn(So,F) < limsup- • 8mlog(#A) = 8rlog(#A). 

In view of Lemma QJii) this finishes the proof . □ 

Of course this result would be false without the factor — in the definition of 
ER„{V,F). 

In order to prove that entropy rate is a natural notion, one must make a 
new assumption: the measure fj, should be shift-invariant, in addition to the 
previous requirement of being f 1 - invariant. Call bi-invariant any measure that 
is invariant both under F and under the group of shifts. 

Proposition 2. Let [i be a bi-invariant measure. For any finite measurable 
partition V of A 1, one has 

ER ll (V,F)<ER ll (S ,F), 

and therefore 

ER^A z2 ,F) = ER^(So,F). 

Proof. Given a finite measurable partition V fix some e > 0. Since the parti- 
tions S' n converge to the discrete partition as n — > oo, the conditional entropy 
Hp(P\S' n ) goes to as n — > oo: choose k such that H,j,(V\S k ) < e. 

From this inequality, keeping in mind that (S' k ) n = S' n+k , one derives another 
one for H^(V n \S' n+k ) in the following way. By definition V n — \J veE ^'{P), so 

H,(V n \S' n+k ) = H,( \/ a v {V)\S' n+k )< ]T H,{a%V)\S' n+k ). 

Note that S' n+k is a refinement of o- v (S' k ) for every v £ E n , because the set Ek + 
v C Z 2 is a subset of E n+k . Thus H^(a v (V)\S' n+k ) < H^(a v (V)\a v (S' k )). Due 
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to the fact that fi is invariant under the shifts, H^{a v (V)\cr v {S' k )) = H^(V\S' k ). 
Then it results from the former majoration of .^('Pnl^+fc) that 

H,{V n \S' n+k ) < £ H^(V\S' k ) < 8rne. 

This, together with inequality [2J allows us to bound the dynamical entropy 
of V n from above: 

K{V n ,F) < h^Vn V S n+k , F) < h^S n+k ,F) + H^{V n \S n+k ) 

< hp(S n+ k,F) +8rne, 

hence 

ER„{V, F) = limsup -h^Vn, F) < limsup -(h„(S' n+k ,F) + 8rne) 

7i— >oo Tl n->oo Tl 

= ER fl (S' k ,F) + 8re. 

Letting e go to (or equivalently letting k go to infinity) this implies that for 
any finite partition V 

ER ll {V,F)<ER lx {S' k ,F). 

As was noted in Proposition [TJ ER^(S k , F) = ER^(So, F) for any k. Since 
So e F(A Z ' 2 ) one thus gets ER^(A z2 ,F) = ER^(S ,F), which finishes the 
proof. □ 

Question 1. Is there exist a CA F , a F -invariant measure \x and a 'partition 
V such that ER^(V,F) > ER fl (S , F) ? 

Remark 1. From the proof of the last Proposition every measure [i which sat- 
isfies the property Ve > , 3k e N such that Vw € Z 2 H ^ (V)\a v {S' k )) < e 
for V E ¥{A z2 ) verifies ER li (A z ",F) = ER^(S ,F). This simple remark al- 
lows us to extend easily the set of probability measures \x where the entropy rate 
equals ER^(So, F) . For instance each measure invariant for the group gener- 
ated by some iterations of the bi- dimensional shift also satisfies ER^(A Z ,F) — 
ERfj,(So, F). 

The last result has two consequences. The first is straightforward: for a CA 
with radius r on the alphabet A and a bi-invariant measure /i, the entropy rate 
of any finite partition is bounded by 8rlog(#A), which is not as obvious as the 
corresponding coarse upper bound for entropy in the one-dimensional setting. 
The second is the following 

Proposition 3. Let {A 1,2 ,F,/j) and (B z2 ,G,v) be two cellular automata en- 
dowed with their respective bi-invariant measures. If there exists a measurable 
map <p: A z — ¥ B z such that 

1. ip commutes with any shift, 

2. ip o F — G o ip and 

3. tpn = v, 
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one has 

ER^(A Z \F) > ER V (B Z \G). 

In particular entropy rate is an invariant for the class of shift- commuting iso- 
morphisms of CA. 

Proof. The proof is an elementary application of the assumptions and of the 
previous results; we give it in some detail in order to show how it relies upon 
the various hypotheses on the isomorphism map <f>. 

Lift any partition V G F(S z2 ) into F(A z2 ) by ip' 1 : then 

(1) H^cp^iV)) = H V {V), since <pn = v; 

(2) this, and the fact that ipoG = Foip, imply that h,j,((p~ l (V),F) = h„(P, G); 

(3) one also has f' 1 ^^) = (<^~ 1 ('P))^ because <p commutes with the group of 
shift. 

Applying (3) to V = Sq(B 1 ' ) and then (2), one gets for any n > r 
h^-\S {B^))' n ),F) =h,{ v -\S' n {B^)),F) =K(S> n {B^),G). 
Carried into the definitions of ER^ and ER V this implies 

ER^(MB^),F) = limsup K^^B^F) 
= limsup^(^))^) = 

n— too Tl 



= limsup WnG) = ER^-\S (B^)) lF ) = ER v (S (B z2 ),G); 

n— >oo rt 

ip is a measurable map, so that Lp~ 1 (So(B z2 )) G F(A z2 ); taking this into 
account, we get ER^A^ ,F) > ER 11 (lp^ 1 (Sq(B 1j2 )), F) = ER U (S (B I ' 2 ),G). 
Since v is a bi- invariant measure from Proposition [5] we obtain ER fJi (A z ,F) > 
ER v {So{B z2 ) 1 G) = ER u (B z2 ,G). 

Finally when (p is an isomorphism the inequality we just obtained applies in 
the two directions and the two entropy rate are equal. □ 

It looks unlikely that one could obtain the same result after relaxing any of 
the invariance or commutation assumptions in this proposition. All of them are 
used somewhere. 

The following result implies that if the measure is bi-invariant, the positivity 
of sequences of type ( —h u (S u ,F)) is independent of the subsequence u n 

\ Un / n£N 

which shows that the definition of the entropy rate seems rather robust. 

Proposition 4. For all two-dimensional cellular automata and bi-invariant 
measure \i one has: 

hm sup — < 8 x hm ml — . 
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Proof. Roughly the proof use the fact that a square E' is a union of 8n squares 
of type E' p without its central part situated at more than r coordinates of the 
near side of the big square. Let (u n )„ e pj and (i> n )neN two sequence of increasing 
positive integers such that 

liminf M*.,*-) = Hm MW) and limSU p = lim 

n u„ n u n 

Fix p > r and to € N such that u m > u p . Putting n = L^pJ we have 

n rn n n 

(SuX+i = V ° (nAs «, V ^ (_n,i) ^ V ° {i,n)s ^ V ^~ n) ^ 

i= — n i=— n i=—n i——n 

It follows that 

n n 

M0S«,) , B+1 ,-F , )< £ M^ (n ' <) 5« p ,i0+ E M* (_fM X.- F ) 

i— — n i— — n 

n n 

i=—n i=—n 

and using the shift invariance of \i we obtain h^{{S Up )' n , x , -F) < 8n • hfj,(S u , F). 
Since (5 Up )^ +1 >: we get 

MgL^) _ „ m < hMs Up y n+1 ,F) < ^mvf) 

u m (n + l)u p ~~ (n + l)u p (n + 1)m p 

Letting to — » oo with n = L^r - ] an d using Lemma [1] which say that h ll {S' nl F) — 
h fJ _(S n ,F) we obtain 

hmsup = Um < 8 . Wfclff. 

n i>„ Up 

Since this last equality is true for all integer p > r we can conclude writing 

h^F) h^S Up ,F) . h^jS^F) 
lim sup < 8 • lim = urn mi . 

n p->oo Up n 

□ 

For all /x-invariant map we have h ll {T k ) — k ■ h^(T). The next result show 
that entropy rate share this property. 

Proposition 5. For all cellular automaton F on A z , k G N and bi-invariant 
measure p, we have ERfj_(A z ,F k ) = k ■ ER fl ((A 1 ' ,F) . 

Proof. From Proposition [2] we only need to show that ER fJi (So, F k ) = k ■ 
ER Pi (Sq, F). Since F is a cellular automaton of radius r we haveV^F-'GSo) < 
Skr and consequently V i Z 1 i ;,_l (>S n ) A S n+ k r - Hence 

,. K{S n ,F k ) h tl {y k i =^F- i {S n ),F k ) h^S n+kr ,F k ) 
hm sup — < hm sup < hm sup — . 

n— too ji — ^-oo Tl n— too ^ 



12 



Since limsup^ M^*) = Umgup^^ M**^.**? = er ^ SqjF ^ and 
since ^(V^Tq 1 F~ l (S n ), F k ) = k ■ h fl (S n ,F) (see [12]) we can conclude writing 

ER(S , F k ) = limsup MV^f-'OSn), F k ) = fc ER ^ F) 

□ 

Remark 2. More generally one can extend Lemma[l\ Proposition^ \3\ and\I)[ 
and obtain similar results for Proposition^ and^4\ using the following definition 
for the d- dimensional case: 

ER^F) = svviER^V^F^V e F(A zd )} 
= sup{limsup — l —h M {P' n ,F)\V G F(A zd )}. 

n— > oo <^ 

In the d dimensional case V' n — \j v ^E' d where E^ is a d dimensional empty 
hypercube of side n and width r . 

3.1 Entropy rate with respect to the partition So, an in- 
variant for continuous and shift invariant isomorphism 



The following sequence of elementary results shows that for all .F-invariant 
measure \i the entropy rate ER fJ/ (So, F) share several properties (but not all) 
with ER IJi (A 1 ' , F) and is an invariant for continuous and shift- invariant isomor- 
phism. Note that we call cylinder any element of a partition S k (fc € N). 

Definition 4. A sliding bock code is a continuous map 4> '■ A 1 — > B 1 such that 
any element a a of the group of the shift on A z there exists <jb the corresponding 
element in the group of the shift on B z such that <fi o a a = o~b ° 4>- 

Proposition 6. For all bi- dimensional cellular automata F and invariant mea- 
sure n one has ER fl (So, F) = svcp CY (F){F>Rfj,(Q, F)} where CY(F) is the set 

of partitions by finite union of cylinders of A z . 

Proof. From the definition of CY(F), for all Q <E CY(F) there exists an integer 
k e N such that Q A S k . It follows that for all n € N we have h^{Q n ,F) < 
h„{S k+n ,F) which implies that ER^Q, F) < ER^S^F) = ER^(S ,F) by 
Lemma [TJ □ 

Remark 3. By the proof of Proposition [3] and Lemma [7] it is straightforward 
that for all bi- dimensional cellular automata F one has 

h„(Q n ,F) \ _ mm f _hM,F) 



ER fl (So,F)— sup < limsup — ■ >= sup limsup 

CY(F) I 71 ) CY(F) 

The next result shows that ER^(Sq, F) is a invariant for continuous and 
shift commuting isomorphism for each -F-invariant probability measure ji. 
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Proposition 7. Let (A z , F, p) and (B z , G, v) be two cellular automata en- 
dowed with their respective invariant measures. If there exists a sliding block 
code ip: A z — > B 1 such that H ll (ip~ 1 (V)) — H^V), ipfi = v and ipfj, = v then 

ER^ (S (A Z2 ), F) = ER V (5 (B z2 ), g) . 

Proof. With these assumptions we can follow exactly the proof of Proposition 
[3] until the argument that tp is a measurable map (line 14 of the proof ) and 
substitute it by ip is a continuous and shift-invariant isomorphism or a one to 
one and onto sliding block code from A z to B z . Using simple compactness 
arguments (see the Curtis Hedlund Lindon Theorem 0]) we can show that 
<^ _1 (<So) C CY(F)) which by Proposition [S] implies that 

ER^ (s (B z2 ),f) > ER^ ((^- 1 (5 )(B z2 ),f) = ER V (s (B z2 ),g) . 

Finally since tp is am isomorphism the inequality we obtained applies in the two 
directions and the two entropies rate are equal. □ 

Note that Proposition [5] is clearly true for ER^(Sq, F) but Proposition 0] 
that gives more meaning to the definition of the entropy rate using a limsup 
requires the shift invariance of the measure fj,. If we compare ER^A^ ,F) with 
ER^So^F) we can say that ER fl (A I ' 2 ,F) is significant when ER^A^.F) = 
ER^(So, F) which is mainly for shift-invariant measure and ER p (So, F) is only 
an invariant for continuous isomorphism. 



4 An upper bound for the entropy rate 

The following basic result for one dimensional CA is similar to several inequal- 
ities (that involved discrete Lyapunov exponents in [5] and (10) ) in the ergodic 
setting but is not written anywhere. 

Proposition 8. When F is a one- dimensional cellular automaton of radius r 
and n is a bi-invariant measure one has the inequality h p (A z , F) < 2r-/i M (A z , a). 

Proof. Let ao be the partition of A z by the central coordinate and a p = 
V" = _„cr _i (ao). Using the fact that /i is a ^-invariant measure and lim p _j. +00 a p 
is the whole Borel a-algebra B we obtain: 

hJF)= lim hJF,a p )= lim lim H ^=o F ~ la v) 

Using the definition of a one-dimensional CA of radius r, for all p G N we can 
state that V^Zq F~ l a p A v£i^.V _i\°~ l { a p) which implies that: 



hJF)= lim hJF,a p )< lim lim ^ 

p— J-+oo p— *--f-oo n— »-foo ji 



It follows that 



h^F) = lim h p (F,a p ) < lim lim 



H v \. v i=-r(n-l) a ~y a Pn 2rn+l- 2r 



p->+oo p->+oo n-y+oo 2rn + 1 — 2r 
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Since (a p ) p gN is a generating sequence for the transformation a and fi a shift- 
invariant measure we can state that for all p € N 



m (vilr^Lx^K)) 



H, 

h u (o~) — h u (a,a v ) = lim 

which allows us to conclude. □ 

The next results can be seen as a two-dimensional analogue of this inequality, 
but its proof is not as simple. It is also a refinement of the coarse upper bound 
in Proposition [T] 

The entropy /i M (A z , a) of the two-dimensional group of shifts for an invari- 
ant measure fi was introduced in As a function of the partitions S n one 
may write it as: 

h {A « j = lim H A V veE ^(S 0) ) = H,(S n ) 
M ' ; n->oo (2n+l) 2 n^oo(2n+l) 2 W 

Theorem 1. Let F be a two-dimensional cellular automaton. If ji is a bi- 
invariant measure on A z one has: 

ER^A 7 ? , F) < 8r x ,a). 

Proof. First we claim that for any n > r the quantity -H^iy 1 -^ F~ l (S' n )\S n - r ) 
tends to a limit as p —¥ oo and that 

p— foo p 

Indeed since the equality 

HMV) = H^V V Q) - H„{V) 
holds for all finite partitions V, Q, one has 

H M ^ V F~ i (S' n )\S n -r \ = H, (\J F-*(S' n ) V S n -}j - H^Sn-r), 

which, taking into account the fact that Vf=o ( S 'n) ^ V?=o ^(^n) v 
S n - r A Vr=o F~ 4 (S„) and dividing by p, yields 

^ i=0 ^ i=0 

1 p ^ 
< ~(£T M ( \/ F-\S n )) - H„(S n . r )). 

P (=0 

Passing to the limit as p — > oo, the term -H fl (S n ^ r ) vanishes, and the lower 
bound and the upper bound converge to h^S^F) and hn(S n ,F) respectively. 
Those two quantities are equal by Lemma [TJ One thus gets 

Hm H^F-^Sn-r) = ( ^ = h ( , ^ 
p— >oo p 
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Equation [5] is proven. 

Applying (jTJi)) iteratively we note that ( Hu(Vf~ F~ % S' |<S„_ r ) ) is a 
subadditive sequence and it follows that 



n p->oo pn nLv^J 

where [^J is the integer part of x £ R. This implies that 

er M ^\f) < limsup ^( v ^ J '^;(^)i^). (7) 

Observe that as F is a cellular automaton of radius r, what happens from 
time to time [v^J — 1 hi the square band E' n is completely determined by the 
coordinates in the square band D n = E' , \ E' , at time 0; in other 
words 

V ^ V ^o)- 

i=0 v£D n 

Recall that So is the partition according to the value of the (0, 0) coordinate. 
The last inequation implies that 

H» (v\^ i - X F- i {S' n )\S n - r 'j < (V veDn a v So)\S n - r ) . 

Put G n = E n+r y^ \ E n - r , then D n = G n U (E n - r \ E n _, r y^)\ injecting this 
into the latter entropy inequality one gets 

H» (^ i ~ lF ' i { S 'n)\'Sn-r) < H„ (V veGn a v <S |«S n _ r ) + 0, 
hence obviously 

H» {y\^- l F-\S' n )\S n . r ) < Hp (V veGn a v S a ) . (8) 

In order to obtain a convenient upper bound for the right-hand term in the 
last inequality, note that as a square band the set G n is the union of 4 rectangles 
of length 2(n + fLv^-J ) + 1 and of width rQv^J + 1)- 

Let x(n) G {0, 1} and k(n) € N be such that ^(Lv^J + l) + xi n ) is odd (when 
r is even x( n ) = L but when r is odd x( n ) varies with n) and k(n) = ^(r(\_T/n\ + 
l)+x[n) — 1). Each of the four rectangles above is covered (not disjointly!) by at 
most [ r ^^t|^|^^ j +1 squares of size f(Lv / "-J + 1) + x( n )i eacri one °f them a 
translate of the square E^ n y A consequence is that the partition \J ' V&G ct v Sq 
is coarser than the supremum of the partitions generated by all coordinates 
belonging to at least one of those squares of size r([^/n\ + 1) + xi n )- 

Since \i is preserved under the group of shifts, for every v € Z 2 one has 
H^{a v (S k ( n) ) = i^(<S fe ( n )). The inequality 



V *W) < 4 (l ^ri^fj J + 1) x H,(s k(n) ) 
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immediately follows, hence by a straightforward computation 



which, passing to the lim sup, implies that 



hmsup^^ * ^ (£) X H ^ 



(9) 



By ([5]), since fc(n) — > oo as n — > oo, 

HfJ. (<5fc(n) / 



,(7) = lim 



M ' ' n^co (2fc(n) + l) 2 ' 

replacing k(n) by its value and carrying the latter inequality into ([3]) one gets 
l imsup ^_# f V g "5 I < limsup-(r(L^j+l)+ X (n)) 2 

= 8rxh^A z ' 2 ,a). 
The desired inequality follows by ([7]) and ((5J): 

ERf,, (A 1,2 ,F)<8rxh fJ , (A 1,2 , a) . 

□ 

Remark 4. Using the more general definition for a d- dimensional CA given in 
Remark^ one can easily extend the last proof (using more complex notations) 
and show that 

ER f ,(A zd 1 F)<(dS r )xh^(A zd ,a) 
where dS r is the surface of the hypercube of side 2r + 1. 



5 Permutative CA 

Computing the entropy h^F) of a cellular automaton is a difficult work in 
general. For some examples it is possible to show that h^(F) = (for instance 
see [10] or [2]). Another exception is the permutative and additive case where 
(see [3]) exact computation is much easier (tractable). In the multidimensional 
case, the entropy of permutative CA is not finite [5]. In this section we show 
how to compute the exact value of their entropy rate in the two-dimensional 
case. 

Recall that patterns and the concatenation PuP' of disjoint patterns P and 
P' are defined in Subsection 12. II 

Definition 5. Let F be a two-dimensional cellular automaton of radius r, let 
f : A^+i) Abe the block map defining F. Choose a position (i,j) G E r . 
The cellular automaton F is called permutative at {i,j) if for any a G A, any 
given pattern P on E r \ {(i,j)} there exists b £ A such that f{P» ({(i,j)} — ► 
b))=a. 
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In other words, F is pcrmutative at (i, j) if, given a pattern P on E r \{(i,j)} 
and some letter a, one can choose a letter b such that a is the output of / for 
P completed by b at 

Let n\ be the uniform measure on A z . By definition for all finite set Eg Z 2 
and element a £ So one has fJ>\(f) veE o~ v a) — A~(# E \ Since #<S = #^4 and 
each element of {\/ vl£E a~ v So) have the same measure we have (see [H]) 

( V °~ VS ^ = * E ■ lo s(#^)- ( 10 ) 

Denote by Pt = {pi,P2,Pz,Pi} the set of points situated at the centers of 
the four sides of the square E r (pi = (0, r), P2 = (0, — r), ^3 = (— r, 0) and 
P4 = (r,0)). 

Proposition 9. // a cellular automata F of radius r is permutative at all the 
points in Pt the entropy of F with respect to the uniform measure fi\ is 

ER^(A z2 ,F) = 8xrx\og(#A). 

Proof. We note that fj,\ is a F-invariant measure since it was shown by Winston 
in |13) that a cellular automaton permutative at only one point p £ E r is 
invariant by the uniform measure. Since p,\ is invariant with respect to the 
group of shift by Proposition [5] we have ER^ X (A z , F) = ER^ X (So , F) . Hence 
using Proposition[T]we can finish the proof showing that for all p £ N and n £ N 
one has 

ER^(A z \F)>8rx\og(#A). 

For all 1 < s < 4 and p £ N let R p s (1 < s < 4) be the four sides of the square 
E' = U* =1 RP. More formally R\ = {(i,j) £ I?\p-r< i < p and -p < j < p}, 
Rl = e I?\ - p < i < p - 1 andp - r < j < p], R p 3 = {(i,j) £ 

I?\~p + r < i < -p and -p < j < p- 1} and R\ = {(i,j) £ Z 2 \~p+1 < i < 
p — 1 and — p + r < j < p}. For all 1 < s < 4 define W s = \J veR p cr~ l So- Using 
the commutativity of F and a we can write that for all positive integer n one 
has 

n-1 n-1 / 4 \ 4 

V = v f-* V = V i^oF-Hn)) . 

z=0 i=0 \s=l / s=l 

Since F is permutative at (r, 0) one has 

n-1 (ri-l)r 
4=0 j=0 

More generaly since F is permutative at (r, 0), (0, r), (— r, 0) and (0, — r) one 

has 

n— 1 ( n ~ l) r (n— l)r (n— l)r (n— l)r 

V^ _< («^)^ V ^ M ' 0) (^i) V °- (0,_i) (ft5) V ^ 0) (^D V 

1 = 4=0 4=0 4=0 4=0 

= V f v 

«=1 \Ra(n,p) / 
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where R\(n,p) = 1? PI {— p + 1 < j < p and p — r < i < p + (n. — l)r}, 
R2(n,p) = Z 2 n {-p + 1 < i < p- 1 and p- r < j < p + {n - l)r}, R 3 (n,p) = 
I?C\{-p < j <p and -p-(n-l)r < i < -p+r} and Ri{n,p) = Z 2 (~}{-p+l < 
i < p and — p — (n — l)r < j < —p + r}. 
From equality [10] we get 

H^V^F^S'p) > H„ x I V I V I I = 8mplog(#A) (11) 

which finish the proof. □ 

Using the same techniques of proof it is possible to compute the entropy rate 
of CA permutative at (J,—r), (— r, k), (r,l) with (— r < i,j,k,l < r). 

Remark 5. Note that if F is permutative at all points in Pt equation ] 11\ implies 
that (i/i M ( l S„, F)) neN i s a converging sequence. Recall that Proposition^brings 
some basic informations about this sequence and the following result show that 
the limit also exists for CA permutative at only two points in Pt C E r . 

Proposition 10. If F is a CA of radius r permutative at (0, r) and (— r, 0) or at 

(r, 0) and (0, — r) then (■^h f±x (S n , F)) ne ^ * s a converging sub-additive sequence. 

Proof. Since F is permutative at (0, r) and (— r, 0) or at (r, 0) and (0, — r) we 
get for all m > |~^] 

Since (i\ is a shift invariant measure we get 

Hp(S n+p ) < H^y^F^Sn) + H^S P ) 
and for all k E N we have 

^(V^oW < H,(v1+™- 2 F-*S n ) + H,(vtoF^S P ) 

which, dividing by k and then letting k go to oo, implies that hp(S„+ p , F) < 
hp(S n ,F) + hp(S p ,F). 

It follows that (^h^Sn, F)) n is a non-increasing sub-additive converging 
sequence □ 

When a CA is not permutative at the four sides of the square E r the cal- 
culous of the entropy rate is more complicated. Nevertheless for the subclass 
of additive CA like for the one dimensional case for the entropy (see [3]) we 
can compute explicitly the value of ERp x (A z , F) and show that the entropy 
is proportional to the number of "additive sites". Note that an additive CA 
is cellular automaton defined thanks to an additive local rule. To simplify the 
notations we will restrict our results to the space {0, 1} Z . 

Calli^i^) the CA defined thanks the local rule /( 12 )(£'r) = ^(Y.o)+ ;r (o.r) mc, d 2, 
^(3,4) the CA defined by /(3,4)(-E r ) — X(_. r! o) +i(o-r) m °d2, -^(1,3) the CA de- 
fined by f(i t 2){E r ) — a:( r .o) + X(o,r) m od2 and Ftu the CA defined by f(i\(E r ) — 
ifrdjmod 2. 
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Proposition 11. We have ER^ lx ({0, l} 2 *, F(i, 2 )) = ER^ ({0, ^(3,4)) 
= £i? MA ({0,1} Z3 ,F (1)3) ) =4rln(2) and ER^ ({0, 1}* F (1) ) = 2r ln(2). 

Proo/. We first show that Fi?^ f{0, F(i,2)) = 4rln(2).We use the same 
notation than in the proof of PropositionlHlwhere ) (1 < s < 4) represent the 
four sides of the empty square E 1 ' and 7£§ = \/ v(£R p cr v (So)- Since 5^ = vf =1 7£f . 
it is easily seen that 

n— 1 ra— 1 4 4 Ai— 1 \ 

v * v *ui>( v ^) = v v W 7 ^) 

i=0 i=0 s=3 s=3 \i=0 / 

(n— l)r (n— l)r 

^ V ^ M ' 0) (^) V ^ (0 -^(^)= V 

i=0 i=0 «e_Ri(n,p)Ui? 2 (n,p) 

Using Equality HU1 we obtain H fl (\/V'=aEZ t 2 jSp) > 4rnplog(#2) which implies 

that ER^ X ^{0, l} z2 , > 4rln(2). To obtain the reverse inequality note 

that since F is permutative at (r, 0) and (0, r) we get for all k € N 

2fc+l 
i=0 

From Lemma [1] and basic properties of the entropy (see [12]) we can assert that 
Vn e N 



/2n+l 



^ (5«,F(i )2 )) < M V %( R "V R 4)-%) = hp (n\jni,F { 



(1,2) 



i=0 



Following the same argument than in the proof of Proposition [T] we get 
ER^ ({0,l} z > (lj2) 



<limsup-^(^\/^^(i,2)) =hmsup-^( \/ (a v (S , F (1 , 2) )) 

n— f 00 ^ n— >-oo ^ ^ n ., inn 

uG-RJ U-RJ 

<limsup- ^^(^o), F(i i2 )) 

<limsup- V ^(^(.So)) < 4rln(2) 



n— >oo 



which finally bring that ER^ X ^{0, l} z2 , i 7 ^ 2 )^ = 4rln(2). Using the same 
arguments for ^(3,4) and P(i,3) we obtain 



ER 



({0,1} Z2 ,F (3 , 4) ) =ER, lx ({0, If 2 , F (li3) ) =4rln(2). 
Using only the side R\ of the empty squares E' p it is easily seen that 
ER^x ({0,lf 2 ,F ( i)) =2rln(2). 



□ 
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6 Topological Entropy rate 



Here we introduce entropy rate in the topological setting. Recall that rele- 
vant properties of the entropy function and of topological entropy are given in 
Subsection 12.21 

Denote by R(A Z ) the set of all finite open covers of A z . In the same way 
as for partitions in Section^ for C € R(A z2 ) put C' n = \J veE , <J V {C) (for n > r) 
and C n = V-ugb aV (C)- Recall that the partitions S n and S' n introduced in the 
same Section are also open covers of the set A 1,2 . 

Definition 6. Let F be a cellular automaton on A with radius r. The entropy 
rate ofCd R(A Z ) is defined as 

ER(C 7 F) = limsup-/i(C' n ,F); 

n— > oo Tl 

The entropy rate of the topological dynamical system (A 1, , F) is the non-negative 
real number 

ER(A Z \F)= sup {ER(C,F)}. 

ceR(A z2 ) 

6.1 First results about topological entropy rate 

Lemma 2. LetU, V be two open covers of A^ withU A V. Then ER(U,F) < 
ER(V,F). 

Proof. For v G Z 2 , owing to the fact that a v is a homeomorphism cr v (U) and 
o- v (V) are also open covers of A z and a v {U) A cr v (V). For n e N it follows 
that U' n A V' n and this implies that h(U' n ,F) < h(V' n ,F), hence ER{U,F) < 
ER(V, F). □ 

The next result is a topological analogue of Lemma Q] together with Propo- 
sition [TJ The proofs are similar. 

Proposition 12. For any cellular automaton F of radius r acting on A z , 
any integer n > r one has h(S n ,F) — h(S' n ,F) and for any n G N one 
has ER(S n ,F) = ER{S Q ,F). Moreover for all k € N we have ER(S k ,F) = 
ER(S Q ,F)<8rlog(#A). 

Proof. Since the topological entropy function H has the same sub-additivity 
property as H^, and since for every finite open cover C one has h(C, F) < H(C) 
(see section 2), we can use the same arguments as in the proofs of Lemma[T]and 
Proposition 3.5 to obtain this result. □ 

The next result is the topological analogue of Proposition [5] 

Proposition 13. For any cellular automaton F on A one has ER(A Z , F) = 
ER(S ,F). 

Proof. The common diameter of the elements of Sk goes to as k — > oo. By 
the Lebesgue Covering Lemma, for any cover C € R(A Z ) there exists a positive 
integer k such that CAS k . By LemmalUit follows that ER(C, F) < ER(S k ,F). 
By Proposition[T2l ER(Sb . F) = ER(Sq,F), which means that any open cover C 
has entropy rate less than or equal to ER(Sq, F). Since So € R(A Z ) the result 
follows. □ 
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The next result shows that since it has the same properties it is possible to 
choose another definition for the entropy rate of an open cover C: ER(C,F) = 
limsupjj^^ i/i(C n , F). Note that since for any n e N C n 9= C' n 

ER{C,F) = limsup -h(C n ,F) > limsup —h(C' n , F) = ER(C,F), 

Note that we have chosen ER(C, F) for its similarity with the measurable case. 
Proposition 14. For any cellular automaton F on A z one has 



sup {ER(C, F)} = sup {ER{C, F)} = ER(S , F) = ER(A , F). 

CGR(A z2 ) CGR(A z2 ) 

Proof. Following the arguments of the proof of Lemma [5] we can assert that 
for any open covers V U one has h(U n ,F) < h(V n ,F), Since for any cover 
C G R(j4 z ) there exists a positive integer k such that C A Sk it follows that 
limsup„_ >0O ±h(C n ,F) < limsup^^ ±h(S n+k ,F) = limsu Ptwoo ±h(S' n+k ,F) 

= ER(S Q , F) = ER(A Z ' 2 , F) from Propositions H and Q3] □ 

Question 2. Is it possible to obtain similar results of Proposition [T^l f or some 
class of non trivial measure in the measurable case ? 

Recall that a sliding block is a continuous map from A z —> 1} that com- 
mute with all shifts. 

Proposition 15. Let (A 1,2 ,F) and (B 1,2 ,G) be two cellular automata. If there 
exists a sliding block code <p : A z — > B 1, such that ip o F — G o ip. then 

ER(A z2 , F) > ER(B z2 , G) . 

More particularly topological entropy rate is an invariant for the class of bijective 
sliding block codes ip : A z — > B z . 

Proof. Since ip o F = G o ip for all n € N and open cover C we have 
h ^ 1 (C n (B z2 )),F) = h(C n (B z2 ),G). 

Moreover since <p> commute with the group of shift one has [</J _1 (iSo)] n — 
^> -1 [(<So)n]' Using Proposition [12] we get 



ER(A z2 ,F) > ER (tp- 1 (s a (B z2 )) ,f) = limsup- 



{[^(So(B^)] ,F) 



lim sup 



hi^p- 1 [(5 (i? z2 )„] ,F) 



limsup h(So{BZ )n ' G) =ER(S Q (B^ 2 ),G). 



Using Proposition rjj which states that ER(B 7? ,G) = ER(S {B z2 ),G) we can 



conclude. 



□ 
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The first part of the following results shows that entropy rate exhibit similar 
properties than entropy and the second gives more meaning to the definition of 
the entropy rate and to the property ER(A Z ,F)=Q. 

Proposition 16. 

(i) For all cellular automaton F on A z and positive integer k we have 

ER{A z \F k ) = k- ER(A Z \F). 

(ii) For all two-dimensional cellular automata one has: 

v HSn,F) , QV . t h(S n ,F) 
hm sup — < 8 x hm mf — . 

n—H>o Tl n^foo n 

Proof, (i) Similar to the proof of Proposition [5] □ 

Proof, (ii) Since by definition F commute with the group of shift h(a v (C), F) = 
h(C, F) for any open cover C. Using this equality we can follows the same proof 
than for the measurable case for shift invariant measure (see Proposition [4]). □ 

6.2 Relation between topological and measurable entropy 
rate 

Proposition 17. Let F be a cellular automaton from A z2 —> A^ 2 . Then 
ER{A z2 ,F)> sup {ER^(A z2 ,F)} and ER(A z2 ,F) > sup {ER^(S , F)} 

fj,eM(F,<r) ' fiGM(F) 

where M(F) is the set of F -invariant measures and M(F,o~) the set of all bi- 
invariant measures. 

Proof. Since each set S n (n € N) is a partition and also an open cover, the 
lowest cardinality of any finite subcover of S n is equal to the cardinality of the 
finite partition S n (N(S n ) = #(S n )). 

Since for all finite partition a and measure /x one has H^(a) < log(#a) (see 
(12) ) we can assert that for all integer p > for all F- invariant measure one has 

^(Vf-Q^-^n) < log (NivtoF-^n)) 

which implies that for all n € N we have h^(S n , F) < h(S n , F) and allow us to 
state the following inequality 

ERJS ,F) < limsup M^lZj = ER(S 0l F) 

n 

that prove the second statement of this Proposition. From Theorem [5] and 
PropositionQJone has ER^ (A 1 - 2 , F) = ER^ (S , F) and ER(S a , F) = ER(A z2 , F) 
which allows to conclude. □ 

It seems not clear if in general there exists some variational principle between 
the topological entropy rate ER(A Z ,F) = ER(S ,F) and ER^(S ,F) (not 
ER^A 1 ,F)) because in order to show that ER^SqjF) > ER(S ,F) we can 
note use classical arguments of the standard variational principle's proof. For 
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instance using some arguments of standard proof of the variational principle 
(see |12j ) we can show that given any open cover /3 there exist a finite partition 
£ and measure [i such that h^^F) > h(/3,F). This can not implies that 
ER(S ,F)>ER I1 (S ,F). 

We believe that the quantity ER(A Z , F) — sup^^^p^ER^^So, F)} rep- 
resents some rate of none scale invariance dynamic for the multi-dimensional 
cellular automaton. Note that this value is equal to zero for permutative CA 
(see Proposition [TS]). 

Definition 7. If an F -invariant measure /i verifies ER(A L , F) = ER I1 (A Z , F) 
we say that fi is a maximum entropy rate measure. 

Proposition 18. The uniform measure on A z is a measure of maximum rate 
entropy for all bi- dimensional cellular automata F permutative at the points 
(0,r), (0,-r). (-r,0) and (r,0). 

Proof. The proof is straightforward from Proposition [5J rT2] and rj3] □ 

Note that the uniform measure is a measure of maximum entropy for one 
dimensional permutative CA. 

Question 3. In fSjj Meyerovitch shows that there exist bi- dimensional CA with 
finite and positive entropy. We wonder if there exists some bi- dimensional CA 
such that h(F) = oo and ER(F) = 0. 

6.3 Entropy rate and CA'extensions 

In the following we give another rather basic argument that underline that the 
notion of entropy rate could be better than entropy to quantify the complexity 
of multidimensional CA. Recall that in dimension one the entropy rate is equal 
to the entropy up to a multiplicative constant. 

In Subsection 12.11 we remind the reader that for any cellular automaton F 
there exists a unique associated block map that defines it completely. In the 
following we show that when a CA acts on a two-dimensional space but its 
block map can be reduced to a one-dimensional one, its entropy rate is equal 
(up to some multiplicative constant) to the entropy of the corresponding one- 
dimensional CA. 

Definition 8. If F is a one- dimensional CA with corresponding block map 
f : A 2r+l — > A; the extension of F to dimension 2 is the two-dimensional CA 
F defined by the local map f : A( 2r+1 ) ->■ A such that for any pattern P on E r 
one has J(P) = f(p), where p = P {0 - r )P(o ,-r+l) ■ • • P (o,r)- 

In other words the local map /, instead of reading all the coordinates in the 
square E r , only reads those of the form (0, i), — r < i < r, and its action is that 
of / on those coordinates. 

Proposition 19. If F is a one dimensional CA and F is its extension to 
dimension 2 one has 

ER(A z \f) = 2 • /i(A z ,F). 
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Proof. From Proposition Q2] one has 

ER(A Z \T) = ER(S ,F) = limsup ^il^lfl 

n— >oo W 

with Co = Sq. For all n g N define a„ = V™ = _„<7 ( ' l ' -'i5>o and note that the open 
cover (which is also a partition) C n = S n = \J r - = _ n o-^°^ s> a n . Using the shift 
commutativity of F we obtain for all k £ N 

^(Vto 1 ^,) = H(V^F-\v^_ n a^a n )) = H(V^_ n a^ (^1^ a n )). 

From the definition of F we can assert that for all i € N one has F^ 1 (a„) = 
" ! 'a„ =^ a n + r i where r is the radius of the CA F. Since for all j E Z — {0} 
one has a^'^a n _L a n it follows that Vfc € N one has 

tf(vfco 1 F _, C B ) = ff(V? = _ n a(°>^=o^n)) - (2n+ ^^(Vto 1 ^ 1 ^) 

which implies that h(C n ,F) = (2n + l)ft.(a„, i* 1 ). Since (a n ) n£ M is a generating 
sequence linin^oo h{a n ,F) = h(F) which allows us to conclude. □ 

Remark 6. When the dimension d > 1 we can use a more general definition 
(likewise that given in Remark\^for the measurable case ) to extend Proposition 
[7PI and show that : 

ER(A Z \F) =2 d ~ 1 -h(A z ,F) 
where F is the extension in dimension d of the one- dimensional CA F. 
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